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Key lerms 


Charter school: A charter school, as defined in this report, is a public school that operates as a school 
of choice as described in the Elementary and Secondary Education Act (ESEA) Section 4310(2). Charter 
schools commit to obtaining specific educational objectives in return for increased autonomy. They are 
exempt from significant state or local regulations related to operation and management but otherwise 
adhere to regulations of public schools. 


Charter school authorizer (Authorizers): An authorized public chartering agency, as defined 

under Section 4310(1) of the ESEA, is a state educational agency, local educational agency, or other entity 
responsible for reviewing and approving or rejecting charter applications and monitoring charter school 
performance related to both academic and fiscal/organizational metrics, as well as compliance with 
relevant laws. State law determines the types and number of organizations permitted to act as authorizers. 


Distress: We define distress in this report as experiencing difficulty achieving or sustaining ESSA’s 
definition of a high-quality charter school. 


Early warning system: This is a process for identifying patterns and characteristics from previous 
events that turned out to be risky, testing those patterns in a local context to identify specific indicators 
and thresholds for risk, and then using the characteristics in a systemic way to identify scenarios of risk 
and to efficiently target interventions. 


Governing board: Sometimes referred to as a school board, this group of individuals serves as a charter 
school’s governing body. The board is ultimately responsible for a school’s quality and performance and 
serves an integral oversight role. A charter contract to operate a school is often held between the authorizer 
and the charter school governing board. 


High-quality charter school: As described in ESEA Section 4310(8) a high-quality charter school 
shows evidence of strong academic results, including significant increases in achievement and/or 
graduation rates where applicable, for any and all subgroups of students, and has no significant issues in 
the areas of student safety, financial and operational management, or statutory or regulatory compliance. 


Indicators of distress: These are characteristics that occur early in a school’s decline suggesting a 
school that is struggling to achieve or maintain high levels of quality. Indicators of distress tend to be more 
difficult to measure but easier to influence than lagging indicators; as a leading indicator, they might 
predict future failure. 


Indicators of failure: These are output data points that occur later in a school’s decline to measure how 
a school performed. Indicators of failure are easier to measure but, as a lagging indicator, require more 
substantive interventions to influence. 


School leader: This term is frequently used in the singular to most often reference a principal, but 
depending on the school, this may be one or more individuals who take on leadership or administrative 
responsibilities for a particular school. 


This term refers to the myriad actors and entities that make up the charter school 
ng but not limited to state education agencies, charter school authorizers, charter school 
zations, governing boards, school leaders, parents, and community members. 
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Background: Integrating Indicators of 
Distress Will Improve Your Portfolio 


Charter schools do not show up at renewal time suddenly failing, 
but rather have likely struggled for some time. By paying attention 
to and addressing signals of school distress before the issues 
permeate all aspects of the school, schools can save their students 
years of lowered outcomes and diminished opportunities. 


Identifying schools in distress affords schools, and the ecosystem 
supporting them, the opportunity to intervene earlier and at the 
cost of fewer resources beyond just funding, including time and 
energy. Although school failure and school improvement are 
complex challenges that look different in various contexts, patterns 
and trends associated with schools in early distress are emerging. 
With this new understanding of stages of school failure-now more 
than ever—we have the opportunity and imperative as a field to 
identify schools in distress while improvement is still feasible. 
Indicators of distress can be identified through the processes 
authorizers already use to monitor and gauge the quality of the 
schools in their sector. 


This publication is the second in a series 
on indicators of distress in charter schools. 
The toolkit builds on the first publication, 
Identifying Indicators of Distress in Charter 


Schools Part 1: The Role and Perspective of 
Charter School Authorizers, which identified 


and described common indicators of distress 


observed by charter school authorizers 

in schools in decline. Future research by 
NCSRC will include examples of how charter 
school authorizers are working with schools 
to address indicators and an analysis of 
indicators of distress as observed by school- 


level leaders. 


Authorizers’ System of Data Collection 


Informal Data 


oe 


Annual Monitoring Renewal 


Collection 
Kel-laiiiniershilelameym lalellersliel Mem DIKiKeSS 


The process of identifying schools in distress does not require vast reams of data or extensive school staff 
time. For many authorizers, identifying indicators of distress in schools simply means looking at existing 
data in a different way. Applying an indicators-of-distress lens to authorizers’ systems of data collection 

provides a structure for authorizers, and other stakeholders, to review data in a longitudinal fashion with 
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How to Integrate Indicators of Distress into Data 
Collection Activities 


Charter school authorizing is inherently contextual work, and each charter school authorizer is the expert 
on their own context and portfolio of schools. Authorizers are encouraged to use the indicators identified 
in Part I and reiterated throughout this toolkit as a starting point, but to also determine which indicators 
characterize struggling schools in their own context. The purpose of this toolkit is to build on the 
process authorizers currently use and support authorizers in identifying struggling schools 
prior to failure, and prior to not meeting renewal criteria. This toolkit walks authorizers through 
auditing the data they are currently collecting and mechanisms for collection to determine whether 
indicators of distress are part of their current efforts. Focusing on these research-based indicators of 
distress, rather than just indicators of failure, such as standardized test scores, means we do not have to 
wait for schools to fall into full organizational failure 


before acting—and more students might be saved from WHAT YOU WILL NEED: 


the negative impacts of a failing or closed school. 


Expertise: These tools can be used individually but are more 
To that end, this toolkit includes tools to help impactful when used with a team. Invite your team members 


authorizers identify indicators of distress, who monitor schools in areas of academic, financial, and 
audit Cicie dats collectonchos for operational performance to participate in the process. 
indicators of distress within their current Information: Each tool will specify the school-specific data 
data collection and monitoring activities, and required, such as annual reports, site visit reports, board 
assess which schools in their portfolio are meeting materials, financial audits, compliance checks, and 
showing indicators of distress and to what data that are informally collected, such as during check-ins 
avinnt: with leaders or during board meetings. 


Tool #3: 
Assessing 


Tool #2: 
Auditing 


Tool #1: 
Identifying 


Tool #1: Identifying Indicators of Distress in Your Portfolio: This tool helps authorizers determine how 


indicators may appear in data and in practice, reflect on these data, and identify indicators of distress in 
their own portfolio. 


Tool #2: Auditing Your Data Collection for Indicators of Distress: This tool supports authorizers in 
reviewing their data collection methods to determine where they are already collecting data on indicators 
and to close gaps to improve the identification of schools in distress. 


Tool #3: Assessing Schools for Indicators of Distress: This tool provides an easy-to-use assessment to 
determine how schools fare on each of the indicators and the extent to which they are showing signs of 
distress. 
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Too! #1: Identitying Indicators of Distress in 
Your Portfolio 


Research has identified seven categories of charter school Purpose: Determine how indicators may appear in data 
core functions, and indicators of distress can arise within and in practice and reflect on indicators of distress in 


: : : Our Own context. 
each one: leadership, governing board, operations, y 


financial management, talent, culture, and instruction.’ Intended Users: Authorizers and other stakeholders 
Indicators do not necessarily show up all at once, in all supporting charter schools: 

categories, and may look different in different contexts. Estimated Time: We recommend 1.5 hours for this 
The type of school, whether the school is independent or activity. How much time you spend is up fo you! 
part of a network, whether the board has been trained, Materials/Data Required: Data collection reports, 
whether the school is in an urban, suburban, or rural area, such as site visits, annual reports, and renewal reports 
and other factors may all affect how indicators become from schools in your portfolio. 

visible. 


The tables below describe the indicators of distress as identified in Part I in each category of charter school 
functions, and the differing ways authorizers and the literature discuss how these indicators may show up 
in practice. A space has been provided for your own notes on how these examples compare to your own 
schools’ experiences of distress. For a more detailed look at each indicator and an indicator assessment for 
schools, see Tool 3: Assessing Schools for Indicators of Distress. 


Instructions: 
1. Think about all the schools that have struggled in your portfolio. 
2. Read through the seven indicators of distress tables below. 


3. In the third column, check the box if struggling schools in your portfolio showed signs of distress that 
were similar to the indicators listed. 


4. In the fourth column, add other examples of related data or indicators that you observed in declining 
schools that do not appear in this table or other thoughts on the relevance of these indicators in your 
own context. 


5. In the reflection section below the tables, reflect on the indicators most relevant in your portfolio and 
why schools in your context may be experiencing these indicators. 


f this categorization can be found in Part 1. 
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Leadership 


Mismatched 
leadership 
competencies 
to context 


Leader's inability to share decision-making authority 


Leader's inability to gain respect/trust of staff or families 


Leader's inability to respect/trust staff 


Leader's failure to demonstrate instructional leadership as 
necessary; for example, by taking minimal interest or time in 
classroom instruction 


Leader's failure to manage time efficiently and appropriately 


Core school operational or instructional tasks drop in quality 


Leader is failing in operational leadership; for example, by 
delegating all business functions to the extent that they 
cannot answer questions about core school operations 


Leader does not demonstrate the ability to make changes, 
handle complaints/concerns, leverage board and staff skills 
and time appropriately 


Leader is unable to create or maintain effective teams 
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Inability 
to sustain 
leadership 


Lack of 
systemic 
leadership 
development 


Lack of 
leadership 


High or increased leader turnover 


Lack of leadership pipeline development 


Lack of succession planning for key leadership and other 


positions 


“Founder's syndrome” demonstrated by a leader who 
develops and starts a school but fails to develop shared or 
distributed leadership structures 


Leader is unable to make decisions 


Lack of academic and organizational leadership 


Decreased frequency or quality of communication with 
authorizer or delays responding to authorizer requests 


Leader is not able to report on key progress indicators 


Leader is not present at the school for a amounts of 
time or to the extent that staff work or school functions are 
negatively impacted 


Leader is not accessible to families, board, or authorizer 
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Governing Board 


Inability to 


iE Failure to recruit or retain sufficient and appropriate 

convene the | membership based on bylaws 

board 
Failure to recruit or retain sufficient depth and diversity of 
expertise necessary for successful board governance 
Low or decreasing attendance at board meetings | | 
Low engagement in the school’s work _a | 
Minimal knowledge of the school's mission, core operations, 
or improvement status 

Board's 

deteriorating Inability to accept feedback from staff or authorizer 

relationship 

ey Decreased frequency or quality of communication with 

SIMO ee authorizer or increased time responding to authorizer 
requests 

Board's 

inability to Poor relationship between board and school leaders 

hold school 

leaders 

accountable | Inadequate review of school leaders’ decisions 
Lack of appropriate structures or tools to execute its strategic _ | 
vision 
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Inadequate 
board 
capacity to 
govern 


Lack of ability to work with school leaders to implement 
changes or improvements as necessary 


Board practices reflect a lack of distinction between 
governance vs. management in charter schools [i.e., board 
attempts to manage the daily operations of a school beyond 
its formal purview} 


Board does not demonstrate strong governance oversight | a | 


Board members fail to engage with the school 


Board members fail to engage with the community 


Board members do not have requisite content expertise 
related to core board functions, such as academics, school 
finances, and school operations 
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Operations 


Breakdown in 
compliance 
and reporting 
functions 


Failure to be 
responsive to 
market needs 


Decision-making that lacks oversight or internal accountability 
among school leadership and/or the board 


Lack of process and procedures for operational decision- 
making among school leadership and the board 


Lack of compliance with relevant reporting requirements 
li.e., applicable laws, rules, regulations, and provisions of the 
sees the school’s authorizer, state education agency, 
district education department, and/or federal authorities 


Missed compliance activities or missed deadlines, such as 
failure to comply with state and local security and building 
safety requirements 


Lack of fulfillment of Generally Accepted Accounting 
Principles (GAAP) 


Evaluation showing the charter holder's annual audit reporting 
packages do not reflect sound operations 


Decrease in student enrollment 


Decrease in student re-enrollment 


Decrease in actual enrollment compared to projected 
enrollment 


Poor reputation in community, decreasing reputation, or lack 
of a reputation 


Poor community engagement policies or practices 
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Finance 


Failure to 
properly 
manage 
finances 


Unclear budgeting/reporting process 


Unaddressed facilities upgrades or repairs 


Debt default [i.e., the school is in default of loan covenani{s] 
and/or is delinquent with debt service payments) 


Poor near-term financial viability: 


+ Falling or variable student enrollment 


« Low current ratio 


+ Low unrestricted days of cash on hand 


Poor financial sustainability measures: 


+ High debt-to-asset ratio 


+ Low or negative cash flow 


2 Authorizers hold slightly different thresholds in determining distress around near-term financial viability and financial sustainability indicators. The bullet points represent 
examples of distress thresholds by one authorizer. 
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+ Low or negative total margin/aggregated 3-year margin. 


- Low debt service coverage ratio 


Misappropri- 


Lack of checks and balances in financial decisions among the 
ation of funds 


board and/or school leadership 


Mismanagement of financial systems 


Hiring of individuals with histories of misappropriation of funds 
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Talent 


Hiring 

challenges Low or decreased student-to-qualified-staff ratios 
Poor/lowered qualifications of hires 
Lack of key personnel to meet student needs |i.e., social 
workers, counselors} 

High teacher 


turnover/Loss_| Falling or variable student enrollment 
of specialized 


High or increased staff turnover 


High turnover or churn of specialized staff, such as special 
education, EL certified teachers 


Staff turnover in the middle of the school year 


Decreasing 
teacher Poor qualifications of hires 


capacity 


Low number of years of experience per hire 


Poor teacher effectiveness 


: 
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Lack of leaders using instruction and school assessment data 
to inform teacher effectiveness supports and professional 
development 
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Culture 


Poor student 
and family 
connection to 
school 


eae or increased parental complaints regarding the school, 
leader, and/or staff 


Lack of meaningful communication with families 


Decreased or low student attendance; low attendance for 
particular student subgroup|s) 


Decreased or low student re-enrollment 


High overall chronic absenteeism; high chronic absenteeism for 
particular student subgroup|s) 


High or increased disciplinary incidents; high or increased 
disciplinary incidents for particular student subgroup\s} 


Inhospitable 
professional _| Staff do not see themselves as part of a team 
culture 


Staff plan and prepare for instruction in isolation 


Decrease in staff meetings/collaboration opportunities 


Decrease in teacher attendance 
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Instruction 


Poor or 
declining 
assessment 
outcomes 


Lack of focus 
on instruction 


Lack of 
cohesion or 
alignment in 
curriculum 


Poor alignment of goals, vision, and mission to meet students’ 
needs 


Lack of differentiation to address specific student needs 


Large or growing gaps between student subgroup outcome 
data [i.e., state assessments} 


k of clear expectations for staff and students 


Lack of teachers using data and school assessment system to 
inform instruction 


Lack of a professional development plan that aligns with the 
program of instruction and best practices 


Insufficient instruction time 


Lack of a systematic process for reviewing and evaluating 

the curriculum, at specific intervals, for alignment with state 
academic standards and improving student academic outcomes 
for the population served 


Poor/lack of alignment of standards, curriculum, assessments 


Teacher pedagogies inconsistent with the school's mission 
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Reflect 


1. While each school in distress is often encountering struggles unique to its own context, there may be indicators that are most relevant (e.g., most 
frequent, most impactful) once you look across the struggling schools in your sector. Review your check marks and notes from the tables above. 
What are the most relevant indicators that schools in your portfolio display—and the evidence exhibited—when experiencing distress? Schools 
will likely exhibit several indicators. Focusing on the most relevant indicators will help you prioritize and address the indicators that are most 
impactful in your portfolio. 


Categories of 
Indicators 


Leadership 


Governing Board 


Operations 


Finance 


Talent 


Culture 


Instruction 


Indicators of Distress Most Relevant to Your Portfolio 


Evidence of Most Relevant Indicators 


= 


Tools to Support Authorizer Data Collection 
SSeS SSS SSS ee 


2. Pertaining to the most relevant indicators noted above, what could be the underlying reasons schools in your portfolio are struggling in these 
areas? As an authorizer, what can you do to address these issues? Indicators idiosyncratic to a particular school may need to be addressed 
specifically with that school, while the same indicators across your portfolio may need to be addressed systemically.’ 


3 This toolkit asks authorizers to reflect on how to support schools in distress, but how authorizers do so varies tremendously based on context. Addressing schools in distress 
will be explored more deeply in upcoming NCSRC publications. 
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Too! #2: Auditing Your Data Collection for 
Indicators of Distress 


Purpose: Reflect on how your current data collection 


Identifying indicators of distress requires that authorizers methods can identify schools showing signs of distress; 
maintain effective and accurate data collection systems determine any duplication and/or gaps in data collection. 
that are usable and useful for both authorizers and Intended Users: Authorizers 

schools. The authorizers interviewed for Part I had Estimated nna: Wercconmcnd sO minulosiecinc 
multiple opportunities throughout their monitoring activity. How much time you spend is up to you! 
aches to collect information on school quality and Materials/Data Required: Data collection instruments, 
determine whether schools were showing signs of distress. such as site visit protocols, annual reports, performance 
Some of these opportunities were more informal and management frameworks, etc. 


occurred on a frequent basis, such as having check-in 

calls with the governing board chair or leadership team or 

reviewing board meeting materials. More formal data collection activities occurred less frequently, such 
as site visits, reviewing annual reports, audits, survey findings, and compliance reports. While the scope, 
frequency, and nature of data collection by authorizers related to identifying indicators of distress varied 
by type and capacity of the authorizer, both formal and informal reviews provided authorizers with ample 
opportunities to collect indicators of distress and identify struggling schools. 


This tool invites you to audit your current data collection efforts and determine the existence and 

extent to which you are currently collecting indicators of school distress. Then consider how you might 
customize your data collection to further gather these data in ways that accommodate your context and the 
requirements of your role as an authorizer, as well as the unique needs of your schools. 


The tables below provide examples of artifacts and sources you may use to identify indicators of distress 
using existing school monitoring processes. While not all the data listed under Artifacts/Sources 

have a bearing on renewal, you may find these data valuable when considering which schools may be 
experiencing distress and where to dig deeper to understand the school’s challenges and to provide 
oversight or support as necessary. 


Instructions: 


1. Read through the seven indicators of distress in the tables below, reviewing the example data artifacts/ 
sources that may be used to track those indicators. 


2. Use the Notes section to list the data you are currently collecting that may also be relevant to these 
indicators. 


3. In the reflection section below the tables, outline any changes needed to your current data collection 
efforts to identify these indicators, such as frequency, type of data, and presentation of data. Keep 

in mind, it is not likely that you will need to revamp your data collection efforts to identify schools 
in deally, you can examine existing data with these indicators in mind and make minor 

s to existing data collection methods. 
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Leadership 


What relevant artifacts/sources are 


Indicator of Distress you currently collecting? Additional notes or artifacts/sources collected 
Mismatched DO Academic data trends 

leadership 

competencies to O Leader evaluation reports 

context 


O Teacher interviews/focus groups 


O Annual report or audit data 


Inability to sustain O Staff retention data 


leadershi 
O Annual report data 


O Interviews or conversations with 
current leaders and board members 


Lack of systemic 0 Staff retention data 


leadership . . 
development O Interviews or conversations with 


current leaders or board members 


Lack of leadership OD Leader interviews 


O Check-ins 
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Governing Board 


What relevant artifacts/sources are 
Indicator of Distress you currently collecting? Additional notes or artifacts/sources collected 


Inability to convene | O) Board meeting observations 


the board 
O Board meeting minutes 


D1 Interviews with board members 


O Review of these data against board 
policies, board bylaws, state open 
meeting laws 


Board's deteriorating |» 1 Check-ins 
relationship with 


authorizer 0 Board minutes 
O Conversations with board members 


O Conversations with staff members 


Board's inability to O Board minutes 


hold school leaders 
accountable O Check-ins 


O Board and school leader evaluations 


Inadequate board 0 Board minutes 
capacity to govern 
0 Board evaluations 
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Operations 


What relevant artifacts/sources are 


Indicator of Distress you currently collecting? 
Breakdown in XO Documentation of process and 
compliance and procedures for operational 
reporting functions decisions 


O Reports to the school’s authorizer, 
state education department, and 
federal authorities: 


O Annual charter report 


O Annual audit reporting 


package 
O Documentation of compliance 
activities 
Failure to be 0 Enrollment data 
responsive to market 


needs O Student retention data 


O Estimated enrollment figures from 
the budget 


OX Parent surveys 


D1 Informal parent and community 
feedback 


O Documentation of community 
engagement policies and practices 


Additional notes or artifacts/sources collected 
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Finance 


Indicator of Distress 


Failure to properly 
manage finances 


Misappropriation of 
funds 


What relevant artifacts/sources are 
you currently collecting? 


O Documentation of internal controls 
D Borrowing history 


O Site visits and/or facility reviews 


O Planning and budgeting 
documentation around facilities 


O Documentation of debt default 


O Annual and renewal reporting/ 
audits 


Financial reporting metrics: 
O Enrollment data 


O Current ratio (current assets divided 
by current liabilities) 


O Unrestricted days of cash on hand 
0 Debt-to-asset ratio 
O Cash flow 


O Total margin (net income divided by 
total revenue)/aggregated 3-year 
margin 

0 Debt service coverage ratio (net 
income + depreciation + interest 
expense}/(principal and interest 
payments] 


O Whistleblower reports or feedback 
from school stakeholders 


O Background checks on school 
leaders and governing board 
members 


O Cash flow 


Additional notes or artifacts/sources collected 
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Talent 


Indicator of Distress 
Hiring challenges 


High teacher 
turnover/Loss of 
specialized staff 


Decreasing teacher 
capacity 


What relevant artifacts/sources are 
you currently collecting? 


0 HR policies and procedures, 
timelines, and strategy 


O Staff roster and student enrollment 
O Open staff positions 
O Staff credentials data 


O Staff retention, annual trends 


O Dates of staff departure 


O Staff credentials data 


O Classroom observations 
D1 Staff interviews/discussions 
O Staff credentials data 


DO Staff support strategy, policies, 
procedures, 


O Staff evaluation policy, procedures, 
staff evaluations 


Additional notes or artifacts/sources collected 


Tools to Support Authorizer Data Collection 


Culture 


Indicator of Distress 


Poor student and 
family connection to 
school 


Inhospitable 
professional culture 


What relevant artifacts/sources are 
you currently collecting? 


O Policy or practices related to family 
engagement, such as a schedule of 
communication with families 


OO Attendance rates 
O Chronic absenteeism rates 


O Disaggregated attendance, chronic 
absenteeism by grade, race/ethnicity, 
socioeconomic status, special 
education, English Learner status, 
and gender 


O Discipline data 


O Disaggregated discipline data, by 
grade in school, fee uy 


socioeconomic status, specia 
education, English Learner status, 
and gender 

OD Student re-enrollment rate 

D1 Staff interviews/discussions 

O Staff attendance 

DO Staff retention 


O Teacher survey data 


Additional notes or artifacts/sources collected 
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Instruction 


Indicator of 
Distress 


Poor or declining 
assessment 
outcomes 


Lack of focus on 
instruction 


Lack of cohesion 
or alignment in 
curriculum 


What relevant artifacts/sources are 
you currently collecting? 


O Progress on a school’s mission- 
specific academic goals 


O Classroom observation data 


O Assessment results, annually and 
trends over time 


O Hees student assessment 
data by grade, race/ethnicity, 


socioeconomic status, special 
education, English Learner status, 
and gender 


O Process of developing professional 
development plan 


O Professional development plan 

O Classroom observation data 

O Teacher interviews and/or surveys 
DO Assessment schedule 


O School schedule 


O Progress on a school’s mission- 
specific academic goals 


O Curriculum and assessment 
documents 


Additional notes or artifacts/sources collected 
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Reflect 


1. Now that you have identified the relevant artifacts and data you are currently collecting, it can be helpful to document which data collection 
activities in your own practice can be used to compile indicators of distress. Informal data collection may occur more frequently, such as 
check-ins with the governing board chair or leadership team, attending board meetings, and reviewing board reports, whereas more formal 
data collection mechanisms can take place annually or less often and tend to have a bearing on renewal decisions. Use the table below to 
indicate where you currently collect information on each indicator. 


Leadership 


Governing 
Board 


Operations 


Talent 


Culture 


Instruction 


Tools to Support Authorizer Data Collection 


2. As an authorizer, your data collection activities serve to monitor and gauge schools 
around renewal criteria. Compiling indicators of distress does not require you to collect EXAMPLES OF DATA COLLECTION 
different data. However, it does ask that you examine the data differently, with an SOURCES 
eye toward decline in the seven research-based areas described above. While not all 
schools will exhibit all indicators, schools that exhibit several of them warrant further 
review. Examining the totality of your data from the tables above, are there 
particular indicator categories or relevant indicators that are not being 


e Board member and school « Site visits 


leader check-ins ¢ Indicators assessment 


« Board meetings = Surveys 


« Board reports/meeting 


accounted for in your data collection activities? List the indicators relevant to Sasi aralk * Compliance reports 
your context that are not accounted for in your data collection activities and identify how - Annual/renewal reporting * Other 
you can collect data on the indicators you identified as missing in your data collection. An and audits 


example row is filled in for you. Data/source examples are included in the sidebar. 


Indicator Data/Source 
Example: Inhospitable - Annual/renewal reporting and audits 
professional culture - Add collection of teacher attendance data 
> Site visits 


- Include question in protocol with staff to gauge professional culture 
» Board reports/meeting materials 
- Review board materials for discussion and strategies related to teacher satisfaction 
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Filling in the Gaps: Formal and Informal 
Mechanisms to Collect Indicator Data 


If there are gaps in your data collection, how could the mechanisms you already use 

be adapted to compile those data in a usable way? Here are some different examples of how 
authorizers collect data on the Leadership indicators in formal and informal ways based on their unique 
authorizing contexts and data collection processes. How you integrate or collect this information will 
depend on your specific processes and relationships with schools. 


Board Member and School Leader Check-Ins: Authorizers can use the reflection questions below to 
guide discussion in formal or informal check-ins with board members and school leaders: 


¢ Can the leader articulate goals and strategies for improvement? 
@ Can the leader and the board articulate a leadership succession plan? 
@ Is the leader often absent from the building? 


@ What is the decision-making process between the school leader and the board for financial and 
operational decisions? Does it seem to be singular or collaborative decision-making? 


@ Does the school leader effectively communicate to the board and the school community? 
@ Is there a clearly understood and written process for how the board is expected to hold the school 
leader accountable? 
Board Reports/Meeting Materials: Authorizers can review board meeting minutes to determine the 
following: 


@ What is the decision-making process between the school leader and the board for financial and 
operational decisions? Does it seem to be singular or collaborative decision-making? 


@ Does the school leader effectively communicate to the board and the school community? How are 
community concerns addressed by the school leader? How is the school leader responding to concerns 
brought forth by the board? 


¢ Are roles clearly defined between the board members and the school leader? Between the school leader 
and other staff? 
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Annual/Renewal Reporting: Here is how one authorizer collects information on leadership sub- 
indicators through their annual reporting rubric. 


Indicator Targets: |s the school leader strong in his or her academic and organizational leadership? 


Does not meet standard = T eader presents concerns in a majority of the sub-indicators with no evidence of a 
n to address the issues. 


Approaching standard eader presents concerns in a minimal number of the sub-indicators and may or may 
credible plan to address the issues. 


Meets standard 1 eader complies with and presents no concerns in the sub-indicators below. 


Exceeds standard «The school leader consistently and effectively complies with and presents no concerns in the 
sub-indicators below. 


Sub-indicators 


Demonstration of sufficient academic and leadership experience Leadership stability in key administrative positions 
Communication with internal and external stakeholders Clarity of roles among school and staff 


Engagement in a continuous process of improvement and establishment | Consistency in providing information to and 
of systems for addressing areas of deficiency in a timely matter consulting with the school's board of directors 


Source: Office of Education Innovation, Indianapolis Mayor's Office 2020-21 Performance Framework 


Indicators Assessment: Under certain circumstances, the authorizer may want to directly assess a 
leader to identify indicators of distress. This could occur through: 


@ Having the school leader or the board fill out the leadership section of the school assessment in 
Tool #3; 


¢ Creating or using a personalized assessment of leadership indicators specific to the authorizing 
context; and/or 


@ Using a leadership evaluation currently in use by the board or the authorizer. 
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Tool #3: Assessing Schools for Indicators of 


Distress 


Examining schools with an eye toward decline may sound 


complex, but most authorizers and schools will find 

they can do so by looking at existing data in new ways. 
This tool distills the indicators of distress and provides 
an opportunity to assess how an individual school is 
performing across a variety of school functions. You 

can use this tool to assess what you know about a single 
school based on a variety of data, or you can choose to 
share this tool with schools in your portfolio to guide the 
schools in reflecting on their performance as preparation 
for a formal review or as a general reflection point. If you 
have completed Tool #1 of this toolkit, you may want to 


Purpose: Assess how a school fares on each of the 
indicators and the extent to which they are showing 
signs of distress. 


Intended Users: Authorizers and other stakeholders. 


Estimated Time: We recommend 30 minutes for this 
activity per school assessed. How much time you spend 
is up to you! 

Materials/Data Required: School-level data, such 

as performance frameworks, annual reports, site visit 
reports, information gathered during check-ins, and 
materials from board meetings. 


focus on the metrics aligned with indicators you identified for schools in distress in your portfolio. 


Instructions: 


1. For each of the following indicators and any additional indicators identified particular to your context, 


rank the school on the scales provided. 


2. Once you have identified where the school falls on the scales, assess how the school scored in each area 


of the seven categories of indicators of distress. 


3. For indicators in which the school scored higher, how can these strengths be leveraged towards the 


school’s success and achievement of desired outcomes? 


In the reflection section below the tables, use the prompts to reflect on the areas where the school scored 
lower or fell in the middle of the spectrum. What steps can be taken to improve? What resources does this 
school need in order to improve? 
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School's Leadership 


Has the respect and trust of the staff. ls not respected by the staff. 


Possesses the competencies to improve the school. Lacks the competencies needed to improve the school. 


Creates an environment where staff have the ability to make and buy in to decisions. Does not allow staff to make decisions. 


Has been stable. Is new. 


Makes timely and appropriate decisions. Struggles to make decisions. 
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School’s Governing Board 


Convenes regularly. Convenes irregularly. 


Shows most members attend board meetings consistently. Shows that a quorum is rarely met. 


Has been able to recruit all its members. Has ongoing open board seats. 


°o oO o 2) oO o) 


ls knowledgeable of the strengths and weaknesses of the school. ls not able to discern the strengths and weaknesses of the school. 


ls capable of leading the work the school needs to do. ls not capable of leading the work the school needs to do. 


Has a board chair who is in regular communication with the authorizer. Has a board chair who refuses to meet with the authorizer. 
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Responds in a timely manner to requests from the authorizer. Does not respond to requests from the authorizer. 


Deliberates openly on decisions. Does not deliberate on decisions; either ignores them 
or just accepts what the leader wants to do. 


Board Chair is in regular communication with the authorizer. Board Chair refuses to meet with the authorizer 


Comprehensively evaluates the school leader on a Does not evaluate the school leader. 
regular basis (at least annually). 


9 °o 3 2 0 oO 


Stays true to its role of governance by focusing on vision and Often manages the school by focusing on the 
strategic leadership. day-to-day decisions. 


Regularly engages with school staff and community. Rarely engages with school staff and community. 
g y €Ngsg y; y €ngsg y; 
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Operations 


There is a process and procedure for operational decision-making There are no processes and procedures, or staff are not 
that is clear and followed by staff. following them to make operational decisions. 


There is a process wherein all operational decisions are checked There is only one person making operational decisions 
by another staff member with authority to supersede the decision. at the school. 


The school complies with federal, state, and local laws, rules, and The school is not in compliance with federal, state, and local 
regulations. laws, rules, and regulations. 


There are few or no errors in submitted reports; data are consistent There are many errors in submitted reports; data are 
across reports. inconsistent. 
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Reports are submitted on time. Reports are often late and/or missing. 


Actual student enrollment numbers are consistently below 
Student enrollment is consistently close to budgeted amounts. budgeted amounts. 


The school does not engage with the community or 
The school regularly and positively engages with the community. does so in a negative manner. 
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Finance 


There is a process and procedure for developing the budget that is There are no clear processes or procedures, or staff are 
clear and followed by staff. not following them to develop the budget. 


The school is not in debt or default. The school has recently defaulted on loan payments. 


The school’s near-term viability is healthy. The school’s near-term viability is declining. 
y Ms y g 


The school’s debt-to-asset ratio is stable. The school’s cash flow is not sustainable. 


All financial decisions are made with board and/or Financial decisions are made by one person at the school 
leadership approval and deliberation. or without board or leadership approval. 


Financial decisions are made in the best interest of the Financial decisions do not align with the best interests 
school's students. of the school's students. 
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Talent 


Staff are teaching in their certified area or area of content expertise. Staff are not teaching in their area of expertise. 


Staff-to-student ratios are appropriate to students’ needs. Staff-to-student ratios are increasing to higher-than-normal levels. 


Staff qualifications match students’ needs. Staff qualifications do not match students’ needs. 


Staff are stable; turnover of staff is minimal. Staff turnover is increasing; personnel leave in the middle of the school year. 


Staff are regularly and comprehensively evaluated. Staff are evaluated only once a year or not at all. 


Staff are engaged in ongoing PD to meet their needs and Staff do not receive PD, or the PD topics are irrelevant 
the needs of students. to the needs of students and staff. 
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Culture 


Student attendance is stable and high; chronic absence is Student attendance is declining; chronic absence is 
low and addressed. increasing and not addressed. 


Student attendance is stable and high across student subgroups. Student attendance is declining or uneven across student subgroups. 


Student behavior infractions are minimal or decreasing. Student behavior infractions are increasing. 


Disciplinary actions are minimal or decreasing. Disciplinary actions are increasing. 


There is no evidence of disproportionate disciplinary actions There is evidence of disproportionate disciplinary actions 
across student subgroups. across student subgroups. 


Staff culture is positive: welcoming, respectful, and supportive Staff culture is diminutive: cliquish and cutthroat among 
among teachers. teachers. 
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Instruction 


The school has clear learning expectations of its students; The school does not have clear learning expectations 
the expectations are high but appropriate. of its students; the expectations are low. 


The school's curriculum and assessments are high-quality The school’s curriculum and assessments lack rigor and 
and aligned with grade-level standards. are not aligned with grade-level standards. 


Students are making progress on the school's mission-specific Students are not making progress on the school’s 
goals. mission-specific goals. 


Students are thriving and showing growth across student Students are not thriving or showing growth across student 
subgroups. subgroups. 
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Additional Measures 


Add Measure: Best Add Measure: Worst 


Add Measure: Best Add Measure: Worst 


Add Measure: Best Add Measure: Worst 


Add Measure: Best Add Measure: Worst 
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Reflect 


Take a moment to review all the responses. What surprised you in your assessment? In which areas did the school receive a high score? Which 


areas need improvement? 


For indicators for which this school received a high score, how can the authorizer and the school leverage these successes? 


Assess the areas where this school scored in the lower or middle portions of the spectrum. What steps can the authorizer and the school take to 


improve these areas? 


What resources does this school need to address areas of weakness? 
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SSeS SSS SSS SSS SS 


Which stakeholders do you need to convene and collaborate with to address areas of weakness? 


How will you hold yourself, and others, accountable for addressing areas of weakness? 
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Conclusion 


Authorizers and other stakeholders in the charter school ecosystem can play important roles in identifying 
schools experiencing longitudinal, deeply rooted distress that might lead to school failure. This toolkit 
provides authorizers with the first steps toward identifying and using early indicators of distress in their 
own contexts to support schools and avoid school failure or closure. 


As you continue to consider how these indicators can be used in your own practice, leverage any 
opportunity to bring together stakeholders across your system to discuss essential questions: 


¢ How will you build in both point-in-time data analysis as well as analyses of trends in school 
performance across the indicators of distress so you can identify both early and sustained distress? 


@¢ What does it mean if a school has been struggling in one category of distress, e.g., leadership, for 
multiple years? 


@¢ What does it mean if a school’s performance on one indicator sharply declines in a single year? 
@ What supports are available and appropriate for schools experiencing distress in a particular indicator? 


Answers to these essential questions rely heavily on the quality and rigor of data collection, your 

team’s professional judgment, and knowledge of the schools in your portfolio. This is where bringing in 
stakeholders with different perspectives will improve your insights into school distress and help to develop 
an appropriate course of action. Along the way, consider how you can build schools’ and other partners’ 
capacities to do this same type of analysis and reflection to help create a true ecosystem of prevention that 
can cooperatively identity schools in distress as early as possible. 


For more on this topic, see our Identifying Indicators of Distress in Charter Schools series, including 


reports and webinars. 


